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Abstract 

This article is devoted to the optimal control of state equations 
with memory of the form: 

x{t) = F (^x{t),u{t), J A{s)x{t - s)ds^ , t>0, 
with initial conditions x{0) = x, x{—s) = z{s),s > 0. 
Denoting by yx,z,u the solution of the previous Cauchy problem and: 

v{x,z) := inf <^ / e''^" L{yx^z,u{^),u{s))ds 

where ^ is a class of admissible controls, we prove that v is the only 
viscosity solution of an Hamilton-Jacobi-Bellman equation of the form: 

Ai'(x, z) + H{x, z, Vxv{x, z)) + {Dzv{x, z), z) = 

in the sense of the theory of viscosity solutions in infinite-dimensions 
of M. Crandall and P.-L. Lions. 
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1 Introduction 



The optimal control of dynamics with memory is an issue that naturally arises 
in many different applied settings both in engineering and decision sciences. 
It is typically the case when studying the optimal performances of a system 
in which the response to a given input occurs not instantaneously but only 
after a certain elapse of time. To cite some recent related contributions, in 
a stochastic framework, we refer to I. Elsanosi, B. 0ksendal, A. Sulem |12] . 
for applications to mathematical finance, and to F. Gozzi and C. Marinelli 
[13] for applications to advertising modelling. In the deterministic case, we 
refer to R. Boucekkine et al. [1] for a generalization of Ramsey's economic 
growth model with memory effects and, in the field of biosciences modelling, 
we refer to the survey of C.T. H. Baker et al. [Ij. 

The aim of the present article is to study, by dynamic programming argu- 
ments, the optimal control of (deterministic) state equations with memory. 
For the sake of simplicity, we will restrict the analysis to (finite-dimensional) 
dynamics of the form: 

P + OO 

x{t) = F ( x{t),u{t), 



/ + 00 \ 
A{s)x{t - s)ds] , t>0, 



with initial conditions x(0) = x and x{—s) = z{s), s > 0. We will also focus 
on the discounted infinite horizon problem: 



v{x,z):=M^J e ^'L{y^^^^u{s),u{s))dsj (1) 

where V is some admissible class of controls. Of course, there are other 
forms of memory effects than the one we treat here: systems with lags or 
with deviating arguments for instance (see for instance [H], [T3], [15] and the 
references therein). 

As is obvious from ([T]), the value function depends not only on the current 
state of the system x but also on the whole past of the trajectory i.e. z 
(note that we have not required z{0) = x in ([T])). Hence the state space 
for problem ([T]) is infinite-dimensional. Note that if we had imposed an 
additional continuity condition ensuring x = z{0), then the value function 
would have been a function of the past z only. There are several reasons why 
we have not adopted this point of view and have prefered to write everywhere 
X and z as if they were independent variables. The main one, is that it enables 
to understand the tight connections between the control problem ([T]) and the 
Hamilton- Jacobi-Bellman equation: 

Xv{x, z) + H{x, z, Vxv{x, z)) + {D^v{x, z),z) = Oii z{0) = x. (2) 
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The previous equation presents several difficulties. The ffist one is of course 
its infinite-dimensional nature. The second one comes from the presence 
of the time derivative of i in the equation and the third one from the 
restriction x = z{0). In a series of articles ( [HI [3, El El [lOl [H]), M. Crandall 
and P.-L. Lions developed a general theory of viscosity solutions in infinite 
dimensions. This theory is of course of particular interest for the optimal 
control of infinite-dimensional systems. In such problems, the Hamilton- 
Jacobi equation frequently contains an unbounded linear term (as in ([2])) 
and in (P, [101 [H]), M. Crandall and P.-L. Lions showed how to overcome 
this additional difficulty. The main contribution of the present paper is to 
show, in a rather simple and self-contained way, how the theory of viscosity 
solutions in infinite dimensions of M. Crandall and P.-L. Lions can be applied 
to fully characterize the value function ([T]) as the unique solution of 1^. For 
the sake of simplicity, we will work in a hilbertian framework i.e. in the state 
space E := X L2(M+,R'^) and defining: 

Eo:={{z{0),z), zeH\R+,R'')}, 

V will be said to be a viscosity subsolution of ([2]) if for every (xq, zq) G M"^ x 
and every (p G C^(M'^ x L^, M) such that v — (p has a local maximum (in the 
sense of the strong topology of M'^ x L^) at {xq, zq), one has: 

Xv{xo, Zq) + H{xo, zo, V^(j){xo, Zq)) + limmf(^^^^)^Eo^{xo,zo) {Dz(l){x, z),z) < 0. 

Supersolutions of ([2]) are defined in a similar way. Now, a convenient way to 
study ([2]) is to rewrite it as an Hamilton- Jacobi equation with an unbounded 
linear term as in M. Crandall and P.-L. Lions P, UHl EI]. Namely, defining 
a = (x, z), the equation reads as: 

Xv{a) + H{a, Vc,v{a)) - x ■ V,,v{a) + {T*{a), Dv{a)) = 0, a G D{T*). (3) 

Where T is the linear unbounded operator on E with domain D{T) = x 
defined by 

Tiy, w) := {y - w{0), -w), V(y, w) G D(T). (4) 
So that its adjoint, T* has domain D{T*) = Eq and is given by 

T*{x, z) := (^(0), i) = (x, i), V(x, z) G D{T*) = Eq. (5) 

Section [2] is devoted to some preliminaries on the Cauchy problem and 
continuity properties of the value function. Section [3] concerns the dy- 
namic programming principle. In section HI we identify the Hamilton- Jacobi- 
Bellman equation of the problem and establish that the value function is a 
viscosity solution of this equation. In section [5l we prove a comparison result. 
Finally, in section [6l we end the paper by some concluding remarks. 
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2 Assumptions and preliminaries 



2.1 On the Cauchy problem 

Let K he a compact metric space, we define the set of admissible controls 
V as the set of measurable functions on (0, +oo) with values in K. For 
z G := L^((0, +cxd), M'^), x G R'' and -u G an admissible control, we 
consider the following controlled equation 

x{t) = F (^x{t),u{t), A{s)x{t - s)ds^ , t > 0, (6) 

together with the boundary conditions: 

x(0) = X, x{—s) = z{s), s > 0. (7) 

In the paper, d and k are given positive integers and we will always assume 
the following on the data A and F: 

• (HI) F G C°(M"' X K X R'',R'^) and there exists a constant Ci > 
such that: 



\F{x,u,a) - F{y,u,(3)\ < Ci{\x - y\ + \a - /?|), (8) 

for every {x, y, a, /3, u) e R'^ x R"^ x R'' x R'' x V, 

• (H2) A G L2((0, +oo), Mfcxd) n L\{0, +oo), M^xd) (M^xd standing for 
the space of real matrices with k rows and d columns). 

In the sequel, we shall sometimes use a stronger assumption than (H2). 
Namely: (H2') A G H\{0, +oo), M,xd) H L\{0, +oo), M,xd). 

Before studying the optimal control of equations with memory of type 
([6]), let us establish the existence, uniqueness and continuous dependence 
with respect to initial conditions for the Cauchy problem The results 

of this section (Propositions [T] and [5]) are fairly standard but we give proofs 
for the sake of completeness and to keep the present paper self-contained. 
Conditions (HI) and (H2) of course ensure existence and uniqueness of a 
solution to the Cauchy problem (in])-©: 

Proposition 1 Assume that (HI) and (H2) hold. For every {x,z,u) G 
R"' X X V , the Cauchy problem admits a unique solution. 
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Proof. For ^ > 0, define 



Ee := {y E C%R+,R'^), snpe-%{t)\ < +00} 

£>0 



and equip Eg with the norm: 



:= supe \y{t)\. 

t>o 

Of course, {Eg, \\.\\g) is a Banach space. For y G Eg, let us define: 

Ty{t):=x+ [ F{y{s),u{s),Gy{s))ds, yt>0 
Jo 

where 

PS /' + 00 

Gy{s) := / A{T)y{s - T)dT + / A{t)z{t - s)dT. 



Until the end of the proof, C will denote a positive constant (only depending 
on F and A) which may vary from one line to another. Let y G Eg, with our 
assumptions on F, we first get: 



\Ty{t)\ < \x\+c[t+^^\\y\\g + j\Gy{s)\d^ . 



(9) 



Now, we also have 



Gy{s)\<( / \A{r)\\y{s - r)\dr ^ \\AU4A 



L2 



1/2 

<\\A\\l2 \\\y\\e( I e^'^^-^Ur] +\\z\\l2 



'26 

Together with ([9]), we then have 

\Ty{t)\e-'' < + C [ te~'' + \\y\\g { ^ + 



which proves that T{Eg) C Eg. For yi and y2 in Eg and t > 0, on the one 
hand, we have: 

\Ty,it) - Ty,{t)\ < C [j-\\y, - y^h + j'^ \Gy,{s) - Gy,{s)\d^ 



on the other hand: 

\GyAs) - GyM\ < -T^Wvi - y2\ 

V Zo 



so that: 

\\Tyi-Ty2\\e<C\\y^-y,\\e(^l + -j^^^^ 

For 6 large enough {6 > 2C + 1, say), T is a contraction of Ee hence admits 
a unique fixed-point. This clearly proves the desired result. |^ 

From now on, for every {x, z, u) & x L"^ x V, we denote by yx,z,u the 
solution of the Cauchy problem ([n])-©- The continuous dependence with 
respect to (x, z) of trajectories of ©-(ITj) is given by: 

Proposition 2 Assume that (HI) and (H2) hold. Let u eV , (xq, Zq) and 
(x, z) he in x L? and define yo := yxo,zo,u, y '■= yx,z,u, then we have 

\yit) - yoit)\ < Ce'\\x - xo\ + \\z - zo\\l^), Vt > 

for some constants C and 9 depending only on F and A. 

Proof. In this proof, C will denote a positive constant that only depends 
on F and A but which may vary from one line to another. Defining for s > 



(3{s):= I A{s - T)y{T)dr + I y4(s + r)z(r)dr, 
Jo Jo 

PS /' + 00 

Po{s):= / A{S - T)yQ{T)dT + / A{s + T)zo{T)dT, 







s 



7(s) := sup - yol, r(s) := / 7, 

[0,s] Jo 

we first have: 

\y{s) - yo{s)\ <\x-Xo\+C (^j^ {\y - yo| + Po\) 

Since we also have 

l/5(r) - /5o(r)| < C {i{r)\\A\\,. + \\A\\,4z - z^,.) 
for t > 0, we then get: 

V\t) < |x - xol + C (r(t) + 11^ - z4L2t) 
which, together with Gronwall's Lemma gives the desired result. 



□ 



Remark 1. Let us remark that when one further assumes that (H'2) holds 
(i.e. A is further assumed to be H^), then the estimate of proposition [2] also 
holds true when one replaces — 2;o||l2 by ||z — 2o||(hi)'- 
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2.2 The optimal control problem 

For L , we consider the optimal control problem 

r+oo 

v{x,z):=mi / e'^'' L{y^^z^u{s),u{s))ds. (10) 

where (H3): A > and L : x K —>■ is assumed to be bounded, 
continuous and to satisfy 

\L{x,u) - L{y,u)\ < C2\x-y\, W{x,y,u) eR'^xW^x K (11) 

for some C2 > 0. Throughout the paper, we will assume that (HI), (H2) 
and (H3) hold. 

2.3 Continuity properties of the value function 

As a consequence of proposition El we deduce that v is bounded and uniformly 
continuous on M'^ x L^, which we denote v G BUC(]R'^ x L^, M). More precisely, 
adapting classical arguments (see e.g. G. Barles ^) to our context, we have: 

Proposition 3 Assume that (HI), (H2) and (H3) hold, then v E BUC(M^x 
L^,]R) and more precisely, defining 9 as in proposition^^ one has: 

1. V is Lipschitz continuous on M'^ x if ^ > 6, 

2. V e CO'"(R"' X L^, M) for every a G (0, 1) tf X = 9, 

3. ve C°'^/^(M'^ X L2,R) ifX<e. 

Proof. Let us define 

5 := \x — Xq\ + \\z — -^oIIl^. 

Let e > and be such that 
/•+00 

/ e'^"" L{y^^z^uXs),Ue{s))ds < v{x, z) + e 
Jo 

setting y^ := y^^^^u,, Vo ■= yxo,zo,u, we then have: 

r+oo 

v{xo,zo) -v{x,z) < / e'^' {L{y^{s),Ue{s)) - L{y%s),Ue{s)))ds + e. 
Jo 
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Using proposition [2] and our assumptions on L, we then get, for some C > 
and all T > 0: 

v{xo,zo) -v{x,z) <C ( [ de'^^-^^'ds + e-^A . (12) 



If A > ^, we then have: 

C6 



v{xo,zo) - v{x,z) < 



X-9 



which proves the first claim. 

If A < ^ and if 5 < 1 (which may be assumed to prove that v is Holder) 
taking e"'^^ := 6^^^ in then yields 

v{xo, Zo) - vix, z)<c(^l + ^ j 6'/' 

which proves the second claim. 

Finally, if A = 6* (and again assuming 5 < 1), taking e"'^"^ = 5 in ( fT2l) 
yields: 

/ ^ log((5) 
v{xo, Zo) - v{x, z) <C { h 5 



A 



which proves the last claim. 



□ 



Remark 2. Again, if A is further assumed to be (i.e. when (H'2) 
holds) then the uniform continuity of v also holds true for the norm [x, z) i-^ 
|x| + i.e. when in the previous proof 6 is replaced by 6 := \x — Xo\ + 

\\z — 2;o||(i^i)/. This fact will be useful later on when proving the comparison 
result. 

In the sequel, we shall denote by C°(M°' x L^,]R) the class of real-valued 
functions defined on R'' x which are sequentially continuous for the weak 
topology of X L"^, we then have the following: 

Proposition 4 Assume that (HI), (H2) and (H3) hold, then v e C°(]R^ x 
Ll,M). 

Proof. Let («„)„, := (z„,x„)„ be a weakly convergent sequence in R*^ x 
and let us denote by a := (x, z) G R'^ x its weak limit. Let it G ^ be 
some admissible control and simply denote i/n := yan,u and y := ya,u 

the 
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trajectories of associated respectively to the initial conditions a„ and a. 
If we prove that y„ converges uniformly on compact subsets to y as tends 
to +00 then the desired result will easily follow from our assumptions on L. 
Let us define 

P+OO P + OO 

5„(t) := / A{t + s)znis)ds, 5{t) := / A{t + s)z{s)ds. 



Jo Jo 
Since A e by (H2), we have: 

\Sn{t)\ < \\A\\L4Zn\\L^<C. 

Thanks to (H2) again, 6n converges pointwise to 6. Rewriting the state 
equation as: 

y{t) = F {y{t),u{t),5{t) + J^Mt- s)yis)ds^ , 
yn{t) = F (^yn{t),u{t), 6n{t) + A{t - s)yn{s)ds 

we get: 

\yn - y\{t) < C (^\yn - ylit) + |5„ - S\{t) + ^ \A{t - s)(y„ - y){s)\ds 

(13) 

(where again in this proof C denotes a nonnegative constant depending only 
on F and A but possibly changing from one line to another). Defining 

7„(t) := sup \yn - y\, r„(t) := / 7n, 
[o,t] Jo 

inequality (fT3l) yields for all s G [0,t]: 

\yn - y\is) < C {\yn - y\{s) + |(5„ - 6\{s) + \\A\\L^Q^t)ln{s)) . 

Integrating the previous yields: 

\yn-y\{s) < \Xn-x\+C (^n{t) + , Vs G [0,t]. 

Hence ^ 

7„(t) = f „(t) < |x„ - x| + C (Vnit) + / % - 5\] . (14) 
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On the one hand, Dominated convergence imphes that 



/• + 00 

hm / \5n - 5| = 0, 
" Jo 



on the other hand, and Gronwall's Lemma imply that r„(t) tends to 
as n tends to +cxo. With (JT^ . this proves that 7n(^) tends to as tends 
to +00 and the desired result follows. Let us remark that, with (fT3l) . this of 
course also implies that {yn)n converges to y in VFj^'^°°(M+, M'^). 



□ 



3 Dynamic programming principle 

Our aim now is to prove that the value function v obeys the following dynamic 
programming principle: 

Proposition 5 Let (x, 2;) G x and t > 0, we then have: 
v{x,z)= inf <^ / e~^''L{y^^^^^{s),u{s))ds + e~^%{y^^,^u{t),yx,z,u{t - ■)) 

{yx,z,u{^ - Ol*) := yx,z,u{^ - ■5) M s > 0). 

Proof. Let e > and G F be such that 

-As 



(15) 



e ''L{yx,z,uX^),u^{s))ds < v{x, z) + e 







we then have 



v{x,z)+e> / e 'L{y^^^^u^{s),u^{s))ds 
Jo 

r+oo 

+e-^' / e-^^L{y,,,,uAt + r),Ue{t + T))dT. 
Jo 

Using the fact that yx,z,uS^ + •) is the trajectory associated to the initial 
conditions {yx,z,uX^), yx,z,ue{t ~ ■)) the control ^^(t + we deduce: 



t 

v{x, z) +e> I e'^'' L{y^^^^uXs) ^Ue{s))ds + e'^%{yx,z,uAt),yx,z,uA^ - •)) 







> inf M e ''L{y^^^^u{s),u{s))ds + e v{y^^^^u{t),yx,z,u{t - ■)) 
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To prove the converse inequality, let u G V, 



£ > and a;^ e y be such that: 



p+oo 

Jo 



defining 



we have 



hence 



Ueis) 



u{s) ifse[0,^] 
LUs{s — t) if s > t 



nt r + oo 

v{x,z)< / e~^^L{y^^^^^{s),u{s))ds+ e~^^L{y^^^^^^^^{s - t),uje{s - t))ds 
Jo Jt 

< / e-^'L{y^,;,,^{s),u{s))ds + e-^\v{xt,Zt)+e) 
Jo 

since u and s > are arbitrary, we get the desired result. 



□ 



4 Viscosity solutions and Hamilton-Jacobi- 
Bellman equations 

4.1 Preliminaries 

Let us define 

Eo:^{{z{0),z), zeH\{0,+oo)X)} (16) 

and remark that Eq is a dense subspace of our initial state space £^ := M'^xL^. 
With the uniform continuity of v on M.'^ x L^, this implies that v is fully 
determined by its restriction to Eq. In fact, we will derive from the dynamic 
programming principle a PDE satisfied by v (a priori only on Eq) and by 
a comparison result we will see that in fact this equation (satisfied in some 
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appropriate viscosity sense) fully characterizes the value function v. Let us 
define 

A{s)z{s)d.s^ , V(x, z, u) eR'^x L'^ X K. (17) 

Before going further, we need the following classical lemma: 

Lemma 1 Let 6 > and z G L^((— 5, +oo), M'^) fort G [0,6], define Zt{s) := 
z{s — t) for s > 0, then Zt converges to z in L^(]R_|_, W^) as t goes to 0+. 

Proof. First, let us remark that ||2;t||L2(K^) converges to ||2;||l2(]j^-) hence it 
is enough to prove that Zt converges weakly to z in L^(R+,]R'^). To see this, 
let / G C°(R+,]R'^), and remark that 

{zt-z)-f= f z{s) ■ f{s + t)ds + / z{s) ■ {f{s + t)- f{s))ds 
J-t 



Since both terms in the right-hand side tend to as f tends to 0, this proves 
that z is the only weak limit point of the bounded sequence zt hence that zt 

+ 1 ■ 



converges weakly to z in L^''™ 



□ 

We will also need the following 
Lemma 2 Let {x, z) G Eq, u eV and for t > define: 

Xt ■= yx,z,u(t), zt{r) := yx,z,u(i - t), Vr > 0, 

then t [xt, Zt) is locally Lipschitz in t (uniformly in the control u E V) for 
the X L^ norm. Moreover, for all t >0 

lim = —Zt in L^, (18) 

s^0+ s 

and, for every t which is a Lebesgue point of t ^ ^{xt, Zt, u{t)) 

lim ^^^"^ ^* = J='{xt, Zt, u{t)). (19) 

Proof. The lipschitzianity of 1 1— > Xt and the proof of ( fT9l) are straightfor- 
ward. To shorten notation, we define y := yx,z,u and remark that for every 
t>0,y e H\{-oo, t),R'^) and that y G W/;^°°'((0, +cx)), M'^) so that (HHD will 
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imply the local lipschitzianity of t t-^ Zt. To prove ffTSj) . let us introduce for 
s > and r > 0: 

A,(r) := ^ + it r) 

s 

which can be rewritten as: 

{y(t — T + a) — y(t — r)) da 

Jensen's inequality first yields: 

A,(t)2 <- [ {y{t -T + a)~y{t~ r)f da 
^ Jo 

using Fubini's theorem, we then get: 

||AJi2<-/ (/ {y{t - T + a) - y{t - T)f dr) da. 
"S Jo \Jr+ J 

By the same arguments as in lemma [T] and since y E //^((— oo, t), R'^), for 
every t > 0, we deduce that for every e > 0, there exists > such that for 
all a G [0, Sg], one has: 

{y{t - T + a) - y{t - T)f dr] <e 



□ 



which proves that converges to in as s goes to 0. 



Remark 3. It follows from lemma [2] that if G C^K x M'^ x L^M) then 
the function t ^ iplt, Xt, Zt) is locally Lipschitz in t (in fact, uniformly in the 
control u eV) and for alH > 0, one has: 

tp{t,Xt,Zt) = 1p{0,XQ,Zo) 

+ / {dt'ilj{s,Xs,Zs) + Vxi>{s,Xs,Zs) ■ J^{xs,Zs,u{s)) - {D^i/jis, Xg, Zs) , Zs)) ds. 
Jo 
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4.2 Formal derivation of the equation 

To formally derive the Hamilton- Jacobi-Bellman equation of our problem, 
let us assume for a moment that v is of class on M'^ x . Let {x,z) G Eq, 
u & K he some admissible constant control, and set 

by the dynamic programming principle, we first have: 

v{x,z)< I e~^'^ L{xs-,u)ds + e~^^v{xt, zt) 
Jo 

so that ^ 

L(x, u) + lim - {e~^^v{xt, zA — v(x, z)) > 
t^o+ t ^ ' 

together with lemma Ej this reads as 

L(x, u) + \J xv{x^ z) ■ jF(x, z, u) — \v{x, z) — {Dzv{x, z), z) > 
and since u is arbitrary, this yields 

Xv{x, z) + {Dzv{x, z), z) + sup{— u) — Vxv{x, z) ■ T{x, z,u)\ < 0. 



We then define the Hamiltonian: 



aix^ z,p) := sup{— u) — p- T{x,z,u)}, \/ix,z,p) eWxL^xW. (20) 

Let u eV, and simply denote {xt^u,Zt,u) ■= {yx,z,u{t),yx,z,u{t - .))• Using 
remark [3 following lemma [21 with x, z) := e~^^v{x, z), we have: 

ft 



e v{xt^u,zt,u) = v{x,z) ~ e " Xv{xs^u, Zs,u)ds 

Jo 

-\- I € x'^i-^SjUi Zs,u) ■ ^s,«) '^(■5)) {Dz'^i-^SjUi Zg^uji Zg^u)") ds 

Jo 

so that the dynamic programming principle yields: 

= inf { / e'^''{L{Xs,u, u{s)) - Xv{Xs,u, Zs,u) + ^xV{Xs,u, Zs,u) ■ J^{Xs,u, Zs,u, u{i 

"GV- Jq 

{'Dz'^(^-^s,uy Zg^u) , Zg^yl^^ds"^ 

> inf { / e'^^i-HiXs^u, Zs,u, ^xV{Xs,u, Zs,u)) - Xv{Xs,u, Zs,u) 
{DzV(^Xs^ii, Zg^n^ y Zg^n'^^ds"^ . 

14 



It is natural to expect the integrand above to converge as t ^ 0^, uniformly 
in u to 

—H{x, z, Vxv{x, z)) — Xv{x, z) — {Dzv{x, z), z) 

so that: 

\v{x, z) + {Dzv{x, z), z) + H{x, z, Vxv{x, z)) > 0. 

Thus, at least formally, the Hamilton- Jacobi-Bellman equation satisfied 
by the value function v can be written as: 

Xv{x, z) + z, V xv{x, z)) + {Dzv{x, z), z) = on Eq. (21) 

with H defined by ([20]). 

The next lemma whose easy proof is left to the reader gives the regularity 
properties of H 

Lemma 3 Let H be the Hamiltonian defined by ^2D\). Assume that (HI), 
(H2) and (H3) hold, then there exists a nonnegative constant C such that: 

\H{x, z,p) - H{y, w,p)\<C {\x -y\ + \\z - wWv^) (1 + b|), (22) 

and 

\H{x,z,p)-H{x,z,q)\ < C\p-q\{l + \x\ + H^IU^), (23) 

for every {x, z,y,w,p,q) G {W^ x L^)^ x R'^ x R''. //, in addition (H'2) is 
satisfied, then l[2^) can be improved by: 

\H{x, z,p) - H{y,w,p)\ <C{\x~y\ + \\z - w^H^y) (1 + \p\), (24) 

for every {x, z,y,w,p) e (R'^ x L^)^ x R'^. 

4.3 Definition of viscosity solutions 

The formal manipulations above actually suggest that the natural definition 
of viscosity solutions in the present context should read as: 

Definition 1 Let w e BUC(M'^ x V, R) f] C°(R'^ x L^, R), then w is said to 
be 

1. a viscosity subsolution of f21\) on x L^ if for every (xq, zq) G x 
and every (p G C^(M'^ x L^,M) such that w — (j) has a local maximum 
(in the sense of the strong topology ofW^ x L'^) at {xq,zq), one has: 

\w{xo, Zo)+H{xo, zo, Va;0(xo, Zo))+\iinmi(^x,z)eEo^{xo,zo) {D^4){x, z),z) < 0, 

where the convergence {x,z) E Eq ^ {xo,zo) has to be understood in 
the strong R'^ x sense. 
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2. a viscosity supersolution of l[21\) on M'^x if for every {xq, Zq) G M^x 
and every (p G C^{W^ x L^, M) such that w — (f) has a local minimum (in 
the sense of the strong topology o/R"' x L"^) at {xo,Zo), one has: 

Xw{xo, zo)+H{xo, zo, V^.0(a;o, zo))+\imsnp(^,^^^-j^E^^^^^^^^^-^ {D^(f){x, z),z) > 0, 

where the convergence {x,z) E Eq ^ {xo,zo) has to be understood in 
the strong M.'^ x sense, 

3. a viscosity solution of l[21\) on M*^ x if it is both a viscosity subsolution 
of (diy and a viscosity supersolution of l[EJ\) on M*^ x . 

4.4 The value function is a viscosity solution 

Proposition 6 Assume that (HI), (H2) and (H3) hold. The value func- 
tion V defined by / flOj) is a viscosity solution of l[21\) on M'^ x L^. 

Proof. Step 1: f is a viscosity subsolution. 

Let ao := {xq, zq) eR'^xL'^ and G C^M^^ x L^, M) such that v{xo, zq) = 
(f){xo, Zq) and > t> on the ball B^- := B{{xq, zq), r) of x L^. Let u E K he 
some constant control For all e > 0, there is some := (x^, z^) G Eq such 
that 

0(ae) - < v{ae) < ^(ae), lima^ = in M"* x 

e 

and such that a^^^ = (x^^^, 2;^ ,,) = (?/a^,u(s), yQ,^_tt(s — .)) belongs to for all 
s G [0,£]. Note that by construction, a^^s belongs to Eq for every s G [0,£:]. 
The dynamic programming principle first yields 

re 



0(ae)-£'< / e-'^U(xe,s,M)cis + e-^^0(a,,e). (25) 

Thanks to the smoothness of lemma [2] and remark [HI we can write: 

e'^^(t){ae,e) = H^e) + £:(Vx.0(ae) ■ J^(a£, u) - X(f){ae)) 



- e~^' {D,4>{ae,s),Ze,s)ds + o{e) 
Jo 

Using ( l25l) . dividing by e and taking the liminf as e ^ 0"*", we then get: 

> liminf£^o+ ^ — J ^^^^L{xe,s, u)ds — Vs0(a£) ■ J^(ae, u) + A0(ae)^ 
1 /"^ 

+ liminf £^0+ - / {D^(f){ae,s) , 4,s) 

^ Jo 
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Since we have: 
1 

liminf£^o+- / (^^^(ae.s), 4,s) ds > liminf(^^^)6£;o^«o {D^(f){x, z),z) , 

^ Jo 

we then obtain: 

> -L(xo,M)-V:r0(ao)-^(ao,M) + Av(ao) + liminf(a,,^)giJo^„(, {D^(f){x, z), z) . 

Since u ^ K is arbitrary in the previous inequahty, taking the supremum 
with respect to u and using the very definition of H given in ( l20i) . we thus 
deduce: 

Xv{ao) + H{ao, V^(p{ao)) + \imM(^^^^)(.Eo^ao {D^(f){x, z),z) < 0, 

which proves that f is a viscosity subsolution of (!2T!) on x L"^. 
Step 2: f is a viscosity supersolution. 

Now let 4> G C'^{W^ X L^, M) be such that w(a;o) = 4>{oto) and t; > on the 
ball Br := -B(q;o, f) of M'^ x L^. For all e > 0, there is some := (x^, G i?o 
such that 

and such that for every u eV, ae,s,u = {xs,s,u, Ze,s,u) = (2/a^,«(s), J/a„«(5 - .)) 
belongs to Br for all s G [0,e]. The dynamic programming principle then 
gives: 

<^(«.) + > mf 1^ e-^'L{xe,s,u,u{s))ds + e-^^0(a,,,,„)| (26) 
With lemma [2] and remark [3l we can rewrite 

+ / e"^'' (Va;(/>(a£,s,«) ■ J^(ae,s,n, u{s)) - {D^(j){ae,s,u),Ze,s,u)) ds 
Jo 

Using fl26|) . we then get: 

> inf { / e~^'(L(x£,^,„, m(s)) - A(/)(ae,^,„) + Va;0(ae,s,«) ■ ^(ae,s,n, u{s)) 
Jo 

- (£'20(ae,3,„),4,s,n))c^s} 

> inf { / e"^''(-i:/'(ae,^,„, Vj,0(ae,s,„)) - A0(a£,s,n) - {D^(f){ae,s,u, Ze,s,u))ds} 
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From the continuity of 'Wx4>i Dzcf)-, and if, we deduce the (uniform in u) 
convergence as e 0+ and s G [0, e], of V x4>{as^s,u)^ Dz4>{ae^s,u) and 
H{as,s,u, Va:0(ae,s,«)) respectively to Va.0(ao), Dz(l){ao) and H{ao, Vx(j){ao)). 
Dividing by —e the last inequality and taking the limsup as e ^ 0^, we thus 
get: 

{D,(P{x,z),z)>0 
which proves that is a viscosity supersolution of fl^Tl) on x L^. 

□ 



5 Comparison and uniqueness 
5.1 Preliminaries 

Our aim now is to prove that v is the unique viscosity subsolution of the 
Hamilton- Jacobi-Bellman equation: 

Xv{x, z) + H{x, z, Vxv{x, z)) + {Dzv{x, z),z) = 0. 

This will of course follow from a comparison result stating that if vi and 
V2 (in a suitable class of continuous functions) are respectively a viscosity 
subsolution and a viscosity supersolution of the equation then vi < V2 on 
E ■=R'^ X L^. As usual, the comparison result is proved, by introducing a 
doubling of variables and by considering perturbed problems of the form: 

sup {fi(ai) - ^2(02) - /^e(ai,a2) : (ai, 02) e (M'^ x L^)^} 

where Pg is some perturbation function (depending on small parameters 9). 
This perturbation includes a penalization of the doubling of variables and 
coercive terms that ensure the existence of maxima say af and Then one 
uses the fact that vi is a viscosity subsolution by taking (j) := Pg{.,a2) as 
test-function. 

To overcome the difficulties due to the fact that the term {Dz<f>{a),z) is 
only defined for z G and that the equation is only justified when in addi- 
tion z{0) = X, one has to be careful on the choice of the perturbation Pg. For 
general infinite-dimensional Hamilton- Jacobi equations with an unbounded 
linear term, these difficulties were solved in a general way by M. Crandall 
and P.-L. Lions in [9l [101 [H]. One of the key arguments of M. Crandall 
and P.-L. Lions in [9j is to use a suitable norm to penalize the doubling of 
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variables. In our context this, roughly speaking, amounts to use a kind of 
{H^y norm instead of the norm in the doubling of variables. As usual, the 
comparison proof will very much rely on the use of quadratic test-functions 
of the form: 

4>{a) = ^i{x) + <l>2(a) + $3(2;) := a|xp + b {B{a - oq), a - ao) + c||2;p 

where a and b and c are constants and 5 is a bounded positive self-adjoint 
operator of M*^ x L^. Let us first note that in our case the term ($3(2;), i) = 
2c {z, z) can be dealt easily since, if z G i/^, one has: 




and since in the definition of viscosity solutions, we have imposed the con- 
vergence of the initial value 2^(0), it is easy to figure out that this term won't 
be a big problem in the proof. The difficulty of dealing with the second term 
{Dz^2{q)-,z) can be solved by properly choosing B as in M. Crandall and 
P.-L. Lions [lO] who emphasized the good properties B should enjoy for the 
comparison proof to work. We now proceed to the explicit construction of 
such a S in our context. 

Let us endow := x with its standard Hilbertian structure, i.e. 
with the norm: 

||q;|P := |xp + ||2||^2, Va = (x, z) E E 

and the corresponding inner product (., .). Let T be the linear unbounded 
operator on E with domain D{T) = R"' x and defined by 

T{y,w) := {y-w{0),-w), y{y,w) G D{T). (27) 

Its adjoint T* has domain D{T*) = Eq = {(x, z) e E : z e H^, z{Qi) = x} 
and is given by 

T*{x, z) := (z(0), i) = (x, i), V(x, z) G D{T*) = E^. (28) 
The unbounded operator / + T*T therefore has domain 

D{T*T) = {{y,w) EE : weH^, y = w{0) - w{0)} 
and is given by 

(/ + T*T){y, w) := {2y - w{0), -iv + w), V(y, w) G D{T*T). (29) 
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Now, let us set B := {I + T*T)-\ For a = {x,z) e E, {y,w) := B{a) is 
defined as follows: firtsly, w G H'^ is the solution of 

-w + w = z in(0,+oo) , - 

-2w{0) + w{0) = X, ^^^^ 

secondly, y is defined by 

X + w(0) , ^ , ^ , ^ 

y = ^ = w{0)-w{0). (31) 

In the sequel we shall also denote B = {Bi, B2) where B2{x, z) = w is defined 
by (IHUI) and Bi{x, z) = y is given by (IHUj) and (IHT]) . Setting: 

:= {B{a), a) , Wa = {x, z) G E, 

and defining w := B2{a) by (IHUl) . an elementary computation shows that 

,, ,,n bp 1^(0)1^ ,, ,,n , , 

ll«lll = ^ + ^^ + 11^11^1 (32) 
which in particular shows that there is some constant C > 0, such that 

\\{x,z)\\l > C [\x\^ + \\z\\\h.)) , V(x,z) G i?. (33) 

Obviously, by construction i? is a self-adjoint, nonnegative compact operator 
on E and TB is a bounded operator on E. For a = {x, z) E Eq = D{T*) 
and w := -82(0), some computations lead to: 

{TB{a),a) = (S(a), r*(a)) = ^\x - wiO)]' + >^\x- ^(0)^, 

and since TB is continuous and Eq is dense in E this proves 

{TB{a),a) >0, ya e E. (34) 
Let us also remark that for a E E and [3 := B{a) one has: 
||a||| = (a,/3) = ((/ + T*T)(/?),/3)>||/?||2, 

so that 

\\B{a)\\ < \\a\\B, Va G E. (35) 

In the sequel we will denote by BUC(Eb,M) the space of bounded and 
uniformly continuous functions on E equipped with the norm Let us 
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remark that because of fl33l) . bounded functions which are uniformly continu- 
ous on E equipped with the usual norm of R'^ x (H^)' belong to BUC(Eb, M). 
In particular if A is then the value function v defined by ( ITOl) belongs to 
BUC(Eb,M). 

We end this paragraph by remarking that equation (I2T1) now can (at least 
formally) be rewritten as 

Xv{a) + H{a, V^t;(a)) - x ■ V^a) + {T*{a), Dv{a)) = 0, a G D{T*). (36) 

As in M. Crandall and P.L. Lions [lOj, we will take advantage of this structure 
(where {T*{a), Dv{a)) has to be understood as {a,T{Dv{a)))) by imposing 
restrictions on test-functions (typically of the form \\a — aolls) rather than 
on a. 



5.2 Comparison theorem 

The comparison result for ( 12T1) then reads as 

Theorem 1 Assume that (HI), (H'2) and (H3) hold. Let vi and V2 be in 

BUC(Eb,M) nC°(E„,M)j respectively a viscosity suhsolution and a viscosity 
supersolution of ^21]) on M.'^ x then Vi < f 2 on R'' x . 

Proof. Let us define M := sups(fi - V2), B := {I + T*T)-^ and 

||a||| := {B{a), a) , \fa = (x, z) e E 

as before. For all e > 0, 5 > and ai := (xi, zi), a2 ■= {x2, Z2), in M°' x L^, 
let us set 6' := (e, 6) and 

$0(01, 02) := Vi{ai) - V2{a2) - - «2||| - ^ + ■ 

Let us also define: 

Me,s = Me = sup {$e(ai, ^2), (ai, 02) e ^ x ^} . 

The weak continuity and boundedness properties of Vi, V2 ensure that the 
supremum M0 is attained at some points = (xf , ) for z = 1, 2. 
Let us set 

$1(0;) := —\\a - aalll + -llalT, Va G E, 

we then have: 

D$i(a) = (^Bi{a - a^) + Sx, ^Bi^a - a^) + Sz^ , Va = (x, z) G E. 
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Since is a maximum of Vi — $i on and since f i is a viscosity subsolution 
of (pri) , we get from definition [TJ 

At;i(ai) + H{al, qe + + \imM^^^^^.^^^E^^^e^ (^B2{a - a^) + 6z, < 0, 

(37) 

where 

qe:= -Biial-a^^). (38) 



We then remark that 



Next, we write: 









1^(0 











X 



{B2{a - a',), z) = {B{a - «^), T*(a)> - Si (a - a^) 

= (TB{a — a^), a) — Bi{a — Oj) ■ x 

and since TB and i?i are continuous, we get: 

{B2{a - al),z) {TB{al - a^), a{) - Bi{a[ - a^) ■ x{ a.s a e Eq ^ q{. 

Hence (1371) can be rewritten as: 



5 1 

\vi{a{) + H{a{,qe + 5x{)-qe-x{~-\x\\^ + -{TB{a\-al),a{) < 0. (39) 

Using in a similar way the fact that V2 is a viscosity supersolution, we arrive 
at: 

5 1 

Xv2{a^2) + H{al, qe - 5x1) - Qe ■ xl + -\xl\^ - - {TB{al - a{),al) > 0. (40) 
Substracting (IH^ and PUI) then yields: 

X{vi{al) - V2{a^2)) + Qe + <^a;?) - H{al qe - (5x^) 

+g, ■ (x^ - X?) - ^(|x?|2 + |a;?n + 1 (TS(a? - a^), a? - a^) < 0. 



Now, thanks to ( IMI) . the last term is nonnegative, which gives: 

X{vi{a'^) - V2{a'2)) + //(a?, qe + Sx^) - H{al qe - 5x1] 

5_ 
2 



+Qe-{A-x{)<^-{\x{\' + \x{\'). (41) 
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By standard arguments (see for instance |j3j or [2J) and thanks to the fact 
that Vi and V2 belong to BUC(Eb,M), one easily obtains: 



lim f , e-^\\al - aUl, Me, vAal) - V2(al)) = (0, 0, M, M). 

*(o+,o+) ' 



On the one hand, using ( !32l) and ( H2l) . we have: 

\x'i - xl\ < V2\\al - al\\B = o{y/e) 



(42) 



on the other hand, using ( l35l) . we have 

\qe\ < -\\B{al - < - a^b = o(^) 

£ B \/ E 

SO that 

lim qe ■ (x?, — x?) = 0. 
9^(0+ ,0+) 

Lemma [3] and (l33l) then imply that there is a nonnegative constant C such 
that: 

|//(a,p) - H{(3,p)\ < C\\a - /9||b(1 + b|), V(a, e x (43) 

and 

\H{a,p) - H{a,q)\ < C\p - q\{l + \\a\\), \/{a,p,q) eExR'^xR'^. (44) 
We thus deduce 

i/(a?, qe + 6x1) " ^(«2, Qe - = H{a1, qe + Sx^) - H{ai qe + Sx^) 
+ H{a% qe + 5x\) - H{al, qe - 5x1) 
<C{\\a[-al\\B{l + \qe\+K\x{\ + \xl\))) 
+ C{{l + \\alm\x\\ + \x',\)) 

= o(v^)o(^) + o(4=)o(v^) as ^ ^ (0+, 0+). 

Putting everything together and passing to the limit in (HTil then yields 
AM < so that the proof is complete. 



□ 

We have already noticed that if A is then v defined by (|TOl) actually 
belongs to BUC(Eb,M). We thus deduce the following 

Theorem 2 Assume that (HI), (H'2) and (H3) hold, then the value func- 
tion V defined by ^^) is the only BUC(Eb,IR) n C°(E„,M) viscosity solution 
of ((UP on M'^ X L^. 
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6 Variants and concluding remarks 
6.1 Finite-dimensional reduction 

For the sake of simplicity, let us assume in this paragraph that d = k = 1 
and that A is an exponential weight: 

A{s) = e'^\ Vs > with 5>0. (45) 

Defining the optimal control problem and its value function v as in fllUj) . 
it is easy to see that, under the special exponential form of A, v actually 
depends on 2; only through the scalar parameter 1/(2;) := e~^^z{s)ds. More 
precisely, setting: 



f (x, z) = w{x, y{z)), W{x, z) E E and y{z) :- 



r+00 

/ e~^'z{s)ds. (46) 
Jo 



it is easy to check that v solves the infinite-dimensional Hamilton- Jacobi 
equation fl?T]) if and only if v is given by fHUl) and w solves the two-dimensional 
equation: 

\w{x, y) + Ho{x, y, d^w{x, y)) - dyw{x, y){5y + x) = 0, (47) 

where 

Ho{x,y,p) := sup{-L{x,u) -p- F{x,u,y)}, y{x,y,p) G 

This finite-dimensional reduction of the problem of course heavily relies on 
the exponential form fH5l) . We refer to p3| for the extension of such finite- 
dimensional reduction in a stochastic setting . 

6.2 The evolutionary problem 

In the present article, we have focused on the stationary case. If we consider, 
the finite horizon optimal control problem 

v{t,x, z) := mf^i^j^ L{s,yt^x^;,^u{s),u{s))ds + g{yt,x,z,uiT))^ (48) 
where yt,x,z,u denotes the solution of the Cauchy problem 



x{s) = F ys, x{s),u{s), J A{t)x{s — T)dT j , t > 0, 
with initial conditions x{t) = x, x{t — s) = z{s), s > 0, 
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this leads to the following evolution equation for v: 

dtv{t, X, z)+ inf {L{t, x, u) + Vxv{t, x, z) • T{t, x, z, u)}—{Dzv{t, x, z),z) — 0, 

together with the boundary condition 

v{T,x,z)^g{x),y{x,z)eE. 
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